How to use the evidence: assessment and application of scientific evidence Handbook series on preparing clinical practice guidelines

The strategic intent of the National Health and Medical Research Council (NHMRC) is to work with others for the health of all Australians, by promoting informed debate on ethics and policy, providing knowledge-based advice, fostering a high quality and internationally recognised research base, and applying research rigour to health issues. NHMRC documents are prepared by panels of experts drawn from appropriate Australian academic, professional, community and government organisations. NHMRC is grateful to these people for the excellent work they do on its behalf. The work is usually performed on an honorary basis and in addition to their usual work commitments. This document is sold through Government Info Bookshops at a price that covers the cost of printing and distribution only. For publication purchases please contact AusInfo on their toll-free number 132 447, or through their Internet address: iii CONTENTS Preface vii INTRODUCTION 1 Development of evidence-based guidelines 1 Using and presenting evidence 1 About this handbook 3 1 ASSESSING THE EVIDENCE 5 1.1 Dimensions of evidence 5 1.2 Level of evidence 5 1.2.1 Study design 6 1.2.2 What if studies disagree? 11 1.2.3 Can causation be established in the absence of randomised controlled trials? 12 1.2.4 Need for different study designs 13 1.3 Quality of the evidence 14 1.3.1 Is study quality important? 15 1.4 Statistical precision of the effect 17 1.5 Size of the effect 17 1.5.1 Measures of treatment effect 18 1.5.2 Interpretation of statistical significance and the size of effect 20 1.6 Relevance of the evidence 23 1.6.1 Appropriateness of the outcomes 24 1.7 Evidence summary 29 1.7.1 Suggested format for an evidence checklist 30 1.7.2 Examples 31 1.8 Conclusion 31 2 APPLYING THE EVIDENCE 33 2.1 Introduction 33 2.2 General issues 34 2.2.1 Can the intervention be reproduced in the setting of interest? 34 2.2.2 Can the results be applied in other settings and to other patients? 35 2.2.3 Can the results be applied to individuals? 36 2.2.4 Using evidence in primary care 36 2.3 Applicability and guideline development 37 2.4 Applying evidence to individuals 40 2.4.1 Transferability 40 2.4.2 Application to individuals 49 iv How to use the evidence